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Abstract. The design of reactive systems must comply with logical cor- 
rectness (the system does what it is supposed to do) and timeliness (the 
system has to satisfy a set of temporal constraints) criteria. In this pa- 
per, we propose a global approach for the design of adaptive reactive 
systems, i.e., systems that dynamically adapt their architecture depend- 
ing on the context. We use the timed automata formalism for the design 
of the agents’ behavior. This allows evaluating beforehand the properties 
of the system (regarding logical correctness and timeliness), thanks to 
model-checking and simulation techniques. This model is enhanced with 
tools that we developed for the automatic generation of code, allowing 
to produce very quickly a running multi-agent prototype satisfying the 
properties of the model. 

Keywords, agent oriented software engineering, formal models, agent 
oriented programming 


1 Introduction 

Real-time reactive syste ms are defined through their capability to continuously 
react to the environment while respecting some time constraints. In a limited 
amount of time, the system has to acquire and process data and events that 
characterize its temporal evolution, make appropriate decisions and produce 
actions. Thus, the robustness of the system relies on its capability to present 
appropriate outputs (logical correctness) at an appropriate date (timeliness). 
Such applications are often critical. Their hardware and software architectures 
have to be specified, developed and validated with care. Then, they are set 
in order for the system to have a detenninist and predictable behavior. The 
interest of multi-agent systems in this context may be considered as limited, 
especially because of autonomy and proactivity properties generally attributed 
to agents. In fact, the decision step in real-time systems is very often hidden and 
examples of usages of multi- agent paradigm in the x cwtiSiC context exploit 

the distributed aspects of multi-agent systems much more than the autonomy 
aspects. 

In this paper, we aim at addressing systems in which time constraints are 
neither critical (obtaining a response a little bit later than specified is accept- 
able) nor strict (when a normal delay of response is exceeded, the result is not 
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immediately worthless but its value decreases more or less quickly with time). 
Another characteristic of such systems is the variability and unpredictability 
of treatments to process and their priority, but also of the availability of ac- 
tive entities (processors) in charge of processing. In such a context of dynamic 
scheduling in distributed systems, there is no solution yet capable to guarantee 
the respect of timing constraints. Our purpose is then to design this scheduling 
so as to optimize the compromise between the respect of logical correctness and 
timeliness, possibly by loosening some constraints when all of them cannot be 
satisfied simultaneously. 

More precisely, rather than scheduling in its classical understanding, our 
concern here is the problem of adaptive reconfiguration of the processing chain 
during the execution. This reconfiguration can occur according to the available 
resources (sensors, processors, effectors), to the wished logical correctness, to 
the measured timeliness and to the events occurring in the environment. But, 
instead of doing this in a centralized manner, the agents will need to control the 
reconfiguration themselves, in addition to their normal activity of data process- 
ing. 

Our objective here is to propose a complete approach, from a software engi- 
neering point of view, for the design of adaptive multi-agent systems. It covers 
all stages of software life cycle, from an abstract specification of the application 
architecture to a testable implementation, including formal verification of prop- 
erties and simulation. The method is based on the formalism of timed automata 
[?], which allows to express systems as a set of concurrent processes satisfying 
some time constraints (section ??). We show that this formalism may be used in 
order to model a multi- agent system from the angle of data processing as well 
as that of dynamic treatment chain reconfiguration (section ??). Then, we show 
how model-checking and simulation may be used to verify selected properties of 
the system and analyze a priori its behavior (section ??). Finally, we address the 
problem of semi- automated translation from a timed automata specification to 
executable agents (section ??). But before giving more details about this work, 
it is necessary to give some words of explanation about our target application 
and its specificities. 

2 Target application and objectives 

The context in which we develop our approach is the project that we call Dance 
with Machine [?]. This project aims at staging a real-time dialogue between a 
human dancer-actor and a multimodal multimedia distributed cognitive system. 
The role of the latter is to achieve in real-time the captation and analysis of 
the performance of the dancer, and to build a multimedia answer to it. This 
answer may consist in visual animations projected on screens around the dancer, 
musical sequences, or actions by robots or other physical objects. We consider 
this application as a metaphorical transposition of the kind of interactions that 
we may forecast between human users and communicating objects. This is called 
Ambient Cognitive Environments (ACE), i.e., physical environments in which 
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perception, processing and action devices have to organize dynamically and in a 
cooperative way in order to provide users with natural interaction and extended 
services. 

The computerized setup is composed of a set of processors equipped with 
communication capabilities. They may also be connected to sensors (video cam- 
eras, biometric sensors, localization sensors, etc.) or effectors (screens, loudspeak- 
ers, engines, etc.). Each processor may run one or several agents, each of them 
being specialized for a specific kind of treatment. Data retrieved from the sensors 
must be handled by several agents before being converted into actions. Agents’ 
work is to analyze, synthesize and transform the data that they get. Data pro- 
duced by an agent are then transmitted to other agents in order to continue the 
processing. The data are finally used to generate pictures, sounds or actions, 
either when the analysis is precise enough, or when the available time is too lim- 
ited. Figure ?? shows a very simplified view of this process. Only one perception 
modality is represented, which corresponds to a video camera. 



Fig. 1. Global architecture of the processing chain in the project “Dance with Machine” . 


The use of agents in this context is justified by the distributed nature of 
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the application (captation, processing and action are distributed among several 
objects and processors). But the main reason why we use agents is to make 
the whole system adaptive in various contexts: when components are added 
or removed, when the global behavior of the system must change, or when time 
constraints are not met by the system. The main time constraint that the system 
should respect is the latency, i.e., the time between the acquisition of data by 
sensors, and the production of corresponding actions by the system, under one 
form or another. This latency should of course be kept as low as possible so that 
the reaction of the system seems instantaneous (at least very quick). On the 
other hand, the analysis of the dancer’s performance should be kept as precise 
and thorough as possible. These two constraints are potentially contradictory 
since a precise and thorough analysis can take significantly more time than a 
rough and superficial one. The quality of an analysis can be measured along two 
complementary dimensions: the precision (for the measure of a parameter of the 
performance) and the thoroughness (when optional treatments are possible, a 
superficial processing will be limited to what is compulsory) . 

Our main purpose is to allow a very quick evaluation of various strategies in 
the control of the processing chain, in order to produce an efficient agent-based 
implementation of the system. We achieve it using a formal model of the sys- 
tem along with tools that we developed to automate the implementation of a 
functional prototype. Model-checking allows to verify that the systems complies 
to the specified constraints (latency, non-blocking, sequentiality of treatments, 
etc.). Simulation, for its part, allows to evaluate the quality of the compromise 
between logical correctness (is the quality of processing satisfactory?) and time- 
liness (does the system comply to time constraints?). 


3 Introduction to timed automata 


Real-time systems may be specified using numerous dedicated methods and for- 
malisms. Most of them are graphical semi-formal notations allowing a state ma- 
chine representation of the behavior of the system. Among the most popular 
formalisms, we may quote Grafcet [?], SA/RT {?], Statecharts [?], UML/RT [?]. 
Such visual representations do not enable to verify the properties of systems and 
it is necessary to associate a formal semantics to them, based in general on pro- 
cess algebras [?], Petri nets [?] or temporal logics [?]. Proposing a new formalism 
is not our intention here. On the contrary, we prefer to examine the potential 
benefit of real-time specification and verification techniques in the design and 
the programming of agent-based reactive systems. We chose for this purpose to 
use timed automata [?]. This formalism has the advantage to be relatively sim- 
ple to manipulate and to possess adequate expressivity in order to model time 
constrained concurrent systems. Moreover, there exists for this model powerful 
implemented tools (e.g., UPPAAL [?]) allowing model-checking and simulation. 
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3.1 Standard model 

A timed automaton is a finite state automaton provided with a continuous time 
representation through real-valuated variables, called clocks , allowing to express 
time constraints. Generally, a timed automaton is represented by an oriented 
graph, where the nodes correspond to states of the system while the arcs corre- 
spond to the transitions between these states. The time constraints are expressed 
through clock constraints and may be attached to states as well as to transitions. 
A clock constraint is a conjunction of atomic constraints which compare the value 
of a clock x, belonging to a finite set of clocks, to a rational constant c. Each 
timed automaton has a finite number of states (locations), one of them being dis- 
tinguished as initial In each state, the time progression is expressed by a uniform 
growth of the clock values. In that way, in a state at each instant, the value of the 
clock x corresponds to time passed since the last reset of x. A clock constraint, 
called an invariant , is associated to each state. It has to be satisfied in order for 
the system to be allowed to stay in this state. Transitions between states are in- 
stantaneous. They are conditioned by clock constraints, called guards , and may 
also reset some clocks. They may also carry labels allowing synchronization. An 
example of timed automaton and a corresponding possible execution is shown 
in figure ??. 



Fig. 2. Example of a timed automaton, where x is a clock. The guard x > 2 and the 
invariant x < 3 imply that the transition will fire after 2 and before 3 time units passed 
in the state. 


The behavior of a complex system may be represented by a single timed 
automaton being a product of a number of other timed automata. The set of 
states of this resulting automaton is the Cartesian product of states of the com- 
ponent automata, the set of clocks is the union of clocks, and similarly for the 
labels. Each invariant in the resulting automaton is the conjunction of the in- 
variants of the states of the component automata, and the arcs correspond to 
the synchronization guided by the labels of the corresponding arcs. 


3.2 Extensions in UPPAAL 

We use UPPAAL for our modelling; a detailed presentation of this tool may be 
found in [?]. We remind here only the main characteristics and extensions with 
respect to the standard model [?]. In UPPAAL, a timed automaton is a finite 
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structure handling, in addition to a finite set of clocks evolving synchronously 
with time, a finite set of integer-valuated and Boolean variables. A model is 
composed of a set of timed automata, which communicate using binary synchro- 
nization through transition labels and a syntax of emission/reception. By con- 
vention, a label fc! indicates the emission of a signal on a channel k. It is supposed 
to be synchronized with the signal of reception, represented by a complementary 
label kl. Absence of synchronization labels indicates an internal action of the 
automaton. The execution of the model starts in the initial configuration (cor- 
responding to the initial state of each automaton with all variable values set to 
zero), and is a succession of reachable configurations. The configuration change 
may occur for three reasons: 

- by time progression corresponding to d time units in the states of the com- 
ponents, provided that all the state invariants are satisfied. In the new con- 
figuration, the clock values are increased by d and the integer variables do 
not change; 

- by a synchronization if two complementary actions in two distinct compo- 
nents are possible, and if the corresponding guards are satisfied. In the new 
configuration, the corresponding states are changed and the values of clocks 
and of integer variables are modified according to the reset and update in- 
dications; 

- by an internal action if such an action of a component is possible, it may be 
executed independently of the other components: the state and the variables 
of the component are modified as above. 

Another peculiarity of UPPAAL, useful in expressing a kind of synchronicity 
of moves, is the notion of “committed" states, labelled in the figures by a special 
label C; see, for instance, the state Choice in the first automaton of figure ??. 
In such a state, no delay is permitted. This implies an immediate move of the 
concerned component. Thus, two consecutive transitions sharing a committed 
state are executed without any intermediate delay. 

UPPAAL allows simulating systems specified in this way, detecting deadlocks 
and to verify, through model-checking, various reachability properties. Typically, 
it can answer the questions like “starting from its initial state, does the system 
reach a state where a given property is satisfied?”, “starting from its initial state, 
is a given property always true?”, or “starting from its initial state, can the 
system reach a given state in a given delay?”. 

4 Modelling a decentralized reactive system 

As stated earlier, timed automata allow to model systems as a set of concurrent 
processes. We will detail gradually in the sequel the way they may be applied 
to our case study. The behavior of our agents consists in receiving and pro- 
cessing input data in order to generate and send new outputs. The processing 
has a duration, considered as fixed, and has to be performed repeatedly. The 
corresponding model is shown in figure ??. 
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agcnt_clk >= min_time 
WorkForAgentN 1 ! 


agent_.dk <= maxjime 


WorkForAgentN? 

lost_data++ 


Fig. 3. A model of a simple agent. 


Initially, the agent is waiting for new data in the state Idle. It starts processing 
on reception of the signal WorkForAgentN passing to the state Processing . It 
comes back to the state Idle at the end of its treatment, which takes a time 
comprised between mm__ time and rnax_ time. The following agent is informed 
then (through the synchronisation on the channel WorkForAgentN 1 ), that it can 
start processing. 

This simple model presents however the following drawback: if a new treat- 
ment request comes to an agent when it is already processing, the corresponding 
data is lost. The number of such events is counted by incrementing the variable 
lost_data. Nevertheless, the loop at the state Processing is necessary to avoid 
deadlocks which may occur if the situation described above happens. A solution 
can be to introduce an additional state playing the role of a buffer (see figure ??). 


Idle 



WorkForAgentN? 
agent_clk 0 


Processing 




Buffer 



agent_.dk <= max_thne 


WorkForAgentN? 

lost_data++ 


agent_clk >= min_time agent_clk >= min_time 

WorkForAgentN 1 ! WorkForAgentN 1 ! 

agent_clk := 0 


Fig. 4. A model of an agent with a buffer. 


Now, if a new request arrives to the agent while it is in the state Processing , it 
passes to the state Buffer. Then, it comes back to the state Processing at the end 
of the treatment, in order to start a next one. If a new request comes when the 
agent is already in the state Buffer , then the corresponding data is lost. At this 
stage, we shall still take into account the fact that a few modules (corresponding 
to various precisions of the processing) are available and may be used to analyze 
the dancer’s posture. A first approach consists in duplicating the agent in charge 
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of the corresponding treatment by associating to each copy a different duration 
constant. However, when a new data is available, it is transmitted to one of the 
agents chosen in a non-deterministic way. Thus, it is necessary to incorporate 
in the agent a controller responsible for choosing between different treatment 
modules. This solution is represented in figure ??. 


Free? 



condi tion_on_agent_cIk 
WorkForModule 1 ! 

Idle Choice / v EndChoice 


Control? 


nee f v 


! condi tion_on_agent_clk 
WorkForModule2! 


EndControl! 


Modulelldle 


WorkForModule 1 ? 
module_clk := 0 


ModulelFree 

<£> 


ModulelProcessing 
module_clk <= max_time 


Free! 


module_clk >= min_time 
WorkForAgentNl ! 


Module2Idle 


WorkForModule2? 
module_clk ;= 0 


Free! 


Module2Free 

<£> 


Module2Processing 
module^clk <= max_time 


module_clk >= min_time 
WorkForAgentNl! 


Fig. 5. A model composed of a generic agent, a controller, module and two treatment 
modules. 


When some data is ready to be processed, the controller module passes in the 
state Choice . The agent chooses to execute a treatment module depending on 
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the value of the boolean expression condition __ on_ agent _ elk When the chosen 
module achieves processing, it informs about it the next agent in the processing 
chain, then it informs the controller by sending the signal Free . 

5 Verification and simulation 

The controller presented in the previous section needs of course to be instanti- 
ated by fixing explicitly the criteria determining the choice between treatment 
modules- We present three different strategies that may be considered and ad- 
dress verification and simulation experiences which may be accomplished for 
some interesting properties. The particular context considered for this study is 
explained in figure ??. 



Fig. 6. A simplified scheme of the processing chain. 


The extraction agent produces an image every 50 ms, which has to be treated 
by the agent in charge of the analysis. This treatment should be performed either 
by a module capable to accomplish a complete analysis or by a module which 
can do only a partial one but taking less time (treatment 2 < t treatment * ) ■ The 
controller has to be designed in such a way that it could be possible to conciliate 
two potentially contradictory criteria: analyzing all images or, in other words, 
avoiding loosing too many of them (timeliness) and performing a maximum of 
complete analyzes (logical correctness). 


5.1 Different strategies of choice 

The first proposal is not really a strategy but we give it as a reference. It consists 
only in systematically alternating the two treatment modules. 

In order to minimize the loss of images, the idea is to anticipate, when the 
agent performs the choice (t choice)^ the date when the agent will receive a new 
image to analyze while it has already an image in its buffer and has not termi- 
nated its current analysis (t Zoss ). This is possible since the frequency of arrivals 
of new images is constant. Thus, in the second strategy, the module 1 will be 
chosen if and only if Ureatmentx < tioss ~ t choice • 

In order to maximize the number of complete analyzes, one can loosen the 
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previous constraint by allowing to use the module 1 even if its execution will 
necessarily entail a loss of an image. In the third strategy, the module 1 will be 
chosen if and only if t treatmenti ^ ( tioss t choice ) * co&f , where co&f fixes the 
limits of allowance. 

5.2 Results 

For each strategy, it is possible to check with UPPAAL that the system satisfies 
certain properties. In particular, we checked that: 

- there is no deadlock: A[ ] not deadlock; 

- there is no image lost: A[ ] lost_data == 0; 

- the ratio of the choice of module 1 is grater than a given threshold: 

A[ ] (nbl * 100 / (nbl + nb2 + Jost_data)) > 50). 

Moreover, it is possible to simulate the system during a given number of cycles 
and to check experimentally the ratio of lost images and images which could be 
analyzed completely versus treatment times t tre atmtnt x and t tr eatment 2 > as shown 
in figure ??. 

Model-checking techniques allow to verify formally and automatically if some 
properties of the system, considered as important, are satisfied in all possible sys- 
tem evolutions. On the other hand, simulation permits to obtain some empirical 
evaluation of performances of the system in terms of logical correctness and 
timeliness, depending on the characteristics of treatment modules and on the 
applied strategy. This allows also envisaging a supplementary control level for 
the agent in charge of the image analysis. This corresponds to a kind of “meta- 
strategy” which could adapt dynamically the strategy of choice depending on 
various constraints and fixed objectives. 

6 Automated code generation 

After having validated the model of the multi -agent system, both formally and 
experimentally, the next stage of development corresponds to translating it into 
an executable prototype. In order to do so, a naive idea could consist in imple- 
menting each timed automaton by a thread, since they are models of concurrent 
processes. Nevertheless, for a same agent modelled by several automata, it could 
involve several synchronization and lead to decline sensibly its performances, 
which could be awkward for a reactive system. Thus, a first step consists in per- 
forming first a synchronized product of all automata describing the same agent 
in order to transform it next into a skeleton of an application. The compiler that 
we developed produces this synchronized product by performing also a number 
of optimizations in order to minimize the size of the resulting automaton. Each 
agent is modelled consequently by a unique timed automaton, which can be 
translated into an executable form in several steps. First, only the finite state 
automaton aspects of the given timed automaton are considered. The states 
where it is necessary to let the time progress are assumed to correspond to some 
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— % module 1 - strategy 1 

% lost data - strategy 1 

% module 1 - strategy 2 

% lost data -strategy 2 

% module 1 - strategy 3 
% lost data - strategy 3 


Fig. 7. The ratio of images analyzed with the module 1 (on the left) and the ratio of 
lost images (on the right), obtained for the second strategy and various values of time 
of treatment for modules 1 and 2. On the bottom, a comparison of the three strategies 
for treatment! = 80ms and coef= 1.25, for various values of treatments - 


treatments. Our compiler translates it in terms of a state in which the agent does 
a break (which is supposed to be replaced by the corresponding treatment mod- 
ule when it is available). Finally, the synchronization signals between automata 
are associated to communications between the corresponding agents. 

7 Conclusion 

We presented in this paper a complete approach, from the software engineer- 
ing point of view, for the modelling of adaptive real-time systems based on the 
multi-agent paradigm. The usage of timed automata specification and verifica- 
tion techniques played here a central and unifying role. We showed how this 
formalism, thanks to its capabilities to model concurrent processes having time 
constraints, can be adapted in order to represent multi-agent systems. Moreover, 
we demonstrated that it could be possible to model in a modular way an agent 
controller, capable to make decisions depending on some fixed objectives. 

The advantage one can take from this formal specification is twofold: First, 
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it is possible to check the model against various kinds of deadlock (or timelock) 
and more generally, against any property coming from a non-respect of time con- 
straints, and avoid this way some problems at a very early stage of development. 
Second, it is worthwhile to take advantage of timed automata representation of 
the system in order to generate automatically application skeletons. To do so, we 
developed a specific compiler which, taking an XML representation of the timed 
automata specification, produces a skeleton based on the JADE multi-agent 
platform [?]. This prototype is finally used to validate choices made previously, 
during modelling and implementation, and to review and modify some of them 
if necessary. 

Finally, the general purpose of this work consists in exploiting the approach 
described in this paper, the design patterns and the composition tools, in order to 
facilitate the design of an entire system. These design patterns could be coupled 
with machine learning techniques for the exploration of parameter spaces, in 
order to optimize agent behaviors when the model becomes more complex. Also, 
it would be interesting to develop an experimental protocol in order to validate, 
on the real prototype, the properties observed on the model. In this context, the 
presented work, even if it is at a preliminary stage, demonstrates however the 
feasibility of this approach and allows to foresee favorably the development of 
powerful and complete tools dedicated to the implementation of reactive multi- 
agent systems. 
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